The present invention relates generally to computer systems and, more particularly, to a method for interacting with a system that includes physical devices interfaced with computer software.
A user who programs systems including physical devices that interface with computer software needs to test and debug various configurations of each system. Such systems typically include one or more physical devices, e.g., switches, sensors, or actuators, that communicate with a computer or a network of communicating computers. The creation, testing, and debugging of computer software is well known to those skilled in the art, but the emerging domain of mixed virtual/real objects is not as well studied.
One particular problem with systems including physical devices that interface with computer software is that the physical devices can be inconvenient to access. For example, the system may include a large number of physical devices scattered throughout a building. This makes the testing and debugging of various configurations of the system awkward and time consuming.
One technique that facilitates working with physical devices is to represent them on the screen as graphical user interface (GUI) elements. Thus, even if the physical devices are remote from the user, a physical device's virtual proxy can be seen on the screen. The use of virtual proxies for physical, i.e., real, devices poses a question regarding locus of control. If the virtual proxy is used to test the physical device, then the state of the virtual proxy no longer reflects the state of the physical device. Especially when the system includes complex physical devices that have multi-dimensional states, the use of a virtual proxy that allows on-screen interaction can cause the user to become quite confused as to the state of the physical device.
One approach to resolve the problem of representing the state of the physical device while enabling virtual interaction with the virtual proxy is to provide two on-screen GUI elements. The first GUI element is a literal proxy that does not allow on-screen interaction; the literal proxy always represents the state of the physical device. The second GUI element is a purely simulated GUI element that allows only on-screen interaction. The second GUI element starts out with a copy of the state of the physical device, but diverges from that state once the user starts to interact with it. In accordance with this approach, the user can use the second GUI element to test and debug the system, and, once satisfied, can disconnect the second GUI element and connect to the first GUI element to get the physical device into the system.
The use of two GUI elements is often satisfactory, but suffers from a number of disadvantages, especially when used in conjunction with more complex systems. First, it requires at least several GUI gestures to replace one GUI element with the other GUI element. This is time consuming and can become quite tedious if the number of connections to and from these GUI elements is large. The process of connecting and disconnecting the GUI elements also introduces the potential for confusion regarding the proper connections to be made. Second, the use of two GUI elements to represent a physical device requires more on-screen real estate. Third, from an aesthetic standpoint, the use of two GUI elements to represent a physical device prevents the on-screen representation of the physical device from looking like the actual physical device.
In view of the foregoing, there is a need for a method for interacting with a system that includes physical devices interfaced with computer software that enables the state of a physical device to be represented while allowing on-screen interaction with the proxy of the physical device.